Points of care in using statistics in method comparison studies.

As clinical chemists and laboratory scientists, we are often concerned when personnel who have little laboratory training begin to perform laboratory tests, such as in point-of-care applications. It may be easy to perform such tests today with modern analytical systems, but there still are things that could go wrong. We hope that some kind of quality system is used to check that everything is working okay with point-of-care analyses. Imagine how statisticians might feel about the powerful statistics programs that are now in our hands. It is so easy to key-in a set of data and calculate a wide variety of statistics—regardless what those statistics are or what they mean. There also is a need to check that things are done correctly in the statistical analyses we perform in our laboratories. In this issue of the Journal, Stöckl et al. (1) provide an interesting discussion of linear regression techniques in method comparison studies, pointing out that the quality of the data may be more important than the quality of the regression technique (e.g., ordinary linear regression vs Deming regression vs Passing-Bablock regression). In this Journal, the standard method for analyzing the data from a method comparison experiment has been to prepare a “comparison plot” that shows the test method results on the y-axis and the comparative method results on the x-axis, and then to calculate regression statistics to determine the best line of fit for the data. Different regression techniques may be appropriate, depending on the characteristics of the data—particularly the analytical range that is covered relative to the test values that are critical for medical applications. Elsewhere in the literature (2), there is a movement to discourage the use of regression analysis altogether and replace it with a simple graphical presentation of method comparison data in the form of a “difference plot”, which displays the difference between the test and comparative results on the y-axis vs the mean of the test and comparative results on the x-axis. This difference plot has become known as the Bland-Altman plot (3). Hyltoft Petersen et al. (4), writing in this Journal, have shown that a difference plot must be carefully constructed to make an objective decision about method performance. The difference plot is actually not so simple when an objective interpretation is to be made. In spite of these recent reports and recommendations on the use of statistics, many analysts and investigators still have difficulties with method comparison data. We studied some of the problems 25 years ago (5) and for the most part, there are similar problems today—with the exception that the calculations are much easier to perform with today’s computer programs. There has not been much improvement, if any, in the basic statistical knowledge and skills available in laboratories today, not only for method validation studies but also for statistical quality control. That does not mean there have not been improvements in the theory and recommendations appearing in the literature, but rather that the practices in laboratories have not really changed very much. Therefore, we still need to exercise a great deal of care in collecting, analyzing, and interpreting method comparison data. Here are some points of care to consider: